1. Field of the Invention
The present invention relates to a frame interpolating method and an apparatus thereof at frame rate conversion in a field of video processing such as MPEG, and more specifically, to a method for determining whether an inputted video is repetitive, and if so, interpolating it to a pixel value in the same location of a previous frame and an apparatus thereof.
2. Description of the Related Art
A frame rate conversion (FRC) means converting the number of frames that are outputted per second. Generally, a frame rate is represented in Hz. For instance, when a motion picture having a frame rate 24 Hz is reproduced through a TV set having a frame rate of 30 Hz, a frame rate conversion is required due to the difference of the frame rates of each video signal. In the case when the frame rate is increased as in the above instance, the frames need to be interpolated.
In a field of video processing such as the Moving Picture Experts Group (MPEG) Standard, the video signal mostly has high autocorrelation to thereby cause redundancy. Hence, effective data compression can be obtained by removing the redundancy. At this time, in order to effectively compress time-varying video frames, it is required to remove the redundancy in two-dimensional space as well as in time.
The removal of the temporal redundancy is based on the idea that the amount of data to be transferred can be greatly reduced by replacing unchanged portions or moved-but-still-similar portions in the frames with their immediately preceding frames.
To do this, it is necessary to find the most similar block between a current frame and a preceding frame, in what is called “motion estimation,” and indicating a displacement of how much the block is moved, i.e., “motion vector.”
A typical method for estimating the motion vectors is a Block Matching Algorithm which takes into consideration precision, efficiency, real-time processability and hardware implementation.
The Block Matching Algorithm compares two consecutive videos such as a previous frame and a next frame, by block units and then estimates a motion based on a degree of matching of signal patterns. In the Block Matching Algorithm, estimation and compensation are carried out in a certain size, that is, in the unit of a set of M pixels in the horizontal direction and N pixels in the vertical direction, and the pixel set is called a “macroblock” which is represented as M×N.
By such Block Matching Algorithm, a motion vector is estimated with reference to the previous frame and the next frame, and a frame interpolation is carried out by using the estimated motion vector.
FIG. 1 is a diagram showing a motion estimation method by using the conventional Block Matching Algorithm. Frame A is a previous frame of an input video, frame C is a next frame of the input video, and frame B is a new frame interpolated using pixel values of the frames A and C. In FIG. 1, t1 indicates a time interval between the previous frame and the interpolated frame, and t2 indicates a time interval between the interpolated frame and the next frame.
Generally, the block matching at frame rate conversion is carried out in such a manner that a search area of the previous frame and the next frame is set based on a block of a certain size, sum of absolute difference (SAD) is compared within the search area, and a matching block between the previous frame and the next frame is found and used as motion information. The SAD is obtained by the equation below.
                    SAD        =                              Q                          i              =              0                                      M              -              1                                ⁢                                          ⁢                                    Q                              j                =                0                                            N                -                1                                      ⁡                          (                                                                                                          f                      2                                        ⁡                                          (                                              i                        ,                        j                                            )                                                        -                                                            f                      1                                        ⁡                                          (                                              i                        ,                        j                                            )                                                                                                  )                                                          [                  Equation          ⁢                                          ⁢          1                ]            
wherein, i, j denote coordinates of a pixel and f (i, j) denotes a pixel value of the pixel located at row i and column j. A subscript of f denotes a relevant frame. Typical search methods for searching the matching block are a full search algorithm and a 3-Step search algorithm. The full search algorithm has high performance but has complex hardware. The 3-Step search algorithm has simple hardware but has low performance. Hence, an appropriate algorithm is adaptively selected according to the desired system configuration.
In the frames having the time interval of FIG. 1, provided that an optimum motion vector obtained through a SAD value between the frame A and the frame C is referred to as MV, a motion vector from the frame B to the frame A is referred to as MVBA and a motion vector from the frame B to the frame C is referred to as MVBC, which are obtained below.
                                                                        MV                BA                            =                                                                    -                                                                  t                        1                                                                                              t                          1                                                +                                                  t                          2                                                                                                      ⁢                  mv                                =                                  [                                                            dx                      BA                                        ,                                          dy                      BA                                                        ]                                                                                                                        MV                BC                            =                                                                                          t                      2                                                                                      t                        1                                            +                                              t                        2                                                                              ⁢                  mv                                =                                  [                                                            dx                      BC                                        ,                                          dy                      BC                                                        ]                                                                                        [                  Equation          ⁢                                          ⁢          2                ]            
wherein, mv indicates unit motion vector [dx, dy].
According to the related art, in the case of a normal video at frame interpolation, a pixel value of the interpolated frame is determined by using the motion vector. That is, the pixel value fB (i, j) of the interpolated frame is obtained below.
                                          f            B                    ⁡                      (                          i              ,              j                        )                          =                                                            t                2                            ×                                                f                  A                                ⁡                                  (                                                            i                      +                                              dx                        BA                                                              ,                                          j                      +                                              dy                        BA                                                                              )                                                      +                                          t                2                            ×                                                f                  C                                ⁡                                  (                                                            i                      +                                              dx                        BC                                                              ,                                          j                      +                                              dy                        BC                                                                              )                                                                                        t              1                        +                          t              2                                                          [                  Equation          ⁢                                          ⁢          3                ]            
In Equation 3, a pixel value of a frame to be interpolated is obtained using the motion vector from the frame B to the frame A and the motion vector from the frame B to the frame C. On the one hand, for fast changing, i.e., fast motion or repetitive motion video, there are difficulties in interpolating unlike normal video because the estimation of motion vector becomes difficult. In this case, according to the related art, the interpolation is carried out by using an average of pixel values of pixels that are in the same locations of the previous frame and the next frame. That is, in the case of such special video, the pixel value of the interpolated frame is obtained below.
                                          f            B                    ⁡                      (                          i              ,              j                        )                          =                                                            t                2                            ×                                                f                  A                                ⁡                                  (                                      i                    ,                    j                                    )                                                      +                                          t                s                            ×                                                f                  C                                ⁡                                  (                                      i                    ,                    j                                    )                                                                                        t              1                        +                          t              2                                                          [                  Equation          ⁢                                          ⁢          4                ]            
In Equation 4, the pixel value fB (i, j) is obtained without using the motion vectors as in Equation 3.
According to the conventional motion estimation methods, the normal video is interpolated using the motion vector, and the special video is interpolated using the average of the previous video and the next video to thereby prevent the severe deterioration of the video. In the case of the fast video, i.e., the video having the fast changing pixel values, blurring of the video may occur in reproducing moving pictures, but the blurring matters little, as it is not very unpleasant to the eye.
However, in the case of repetitive video, blurring of the video occurs between the inputted video and the interpolated video. In addition, when watching the motion pictures, a luminance difference between a light area and a dark area is unpleasant to the eye.